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Art Unit: 2626 

DETAILED ACTION 

1 . In view of the Appeal Brief filed on 2/1 3/06, PROSECUTION IS HEREBY 
REOPENED. As set forth below. 

To avoid abandonment of the application, appellant must exercise one of the 
following two options: 

(1) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply 
under 37 CFR 1.1 13 (if this Office action is final); or, 

(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41 .31 followed 
by an appeal brief under 37 CFR 41 .37. The previously paid notice of appeal fee and 
appeal brief fee can be applied to the new appeal. If, however, the appeal fees set forth 
in 37 CFR 41 .20 have been increased since they were previously paid, then appellant 
must pay the difference between the increased fees and the amount previously paid. 

A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by 
signing below: 



Claim Objections 

2. Claim 30 is objected to under 37 CFR 1 .75(c), as being of improper dependent 
form for failing to further limit the subject matter of a previous claim. Applicant is 
required to cancel the claim(s), or amend the claim(s) to place the claim(s) in proper 
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dependent form, or rewrite the claim(s) in independent form. Claim 30 repeats the 
limitations given in its parent claim 19. 

Claim Rejections - 35 USC §112 

The following is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

3. Claim 21 is rejected under 35 U.S.C. 112, second paragraph, as being indefinite 
for failing to particularly point out and distinctly claim the subject matter which applicant 
regards as the invention. There is not antecedent basis for "the time increments". 

The following rejections are made by interpreting the above claims as taught by 
the art used in the rejection. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 
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4. Claims 1,7, 8, 10, 11, 13, 27 and 28 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Mault et al. (U.S. Patent Application Publication 2001/0049470 A1), 
hereinafter referred to as Mault. 

Regarding claiml, Mault discloses an activity-monitoring device using speech 
recognition that includes the following: 

• accessing speech data (Fig. 4, Audio Input, item 95); 

• recognizing at least two voice commands from the speech data, each voice 
command occurring at a different time fl|[0061], "begin swimming," "end swimming"); 

• determining a first time associated with a speaking of a first of the voice commands, 
wherein said first voice command identifies a start of a time interval flj[0061], 
Timestamps); and 

• determining a second time associated with a speaking of a second of the voice 
commands, wherein said second voice command identifies an end of said time interval 
(11[0061] "end swimming" timestamps); and 

• storing data identifying said time interval and data identifying one or more of said 
first voice command and second voice command fl|[0032] memory to record data; 
1f[0061] recording of speech and timestamps). 

Regarding claim 7, Mault teaches everything claimed, as applied above (see 
claim 1). In addition, Mault teaches: 

• recording speech onto a portable recorder (Fig. 1, ffl0017" audio recording); and 
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• loading the speech data from the portable recorder to the computer system, to the 
speech data comprising the speech and a plurality of time stamps (Fig. 1 , lf[0017] 
download audio files). 

Regarding claim 8, Mault teaches everything claimed, as applied above (see 
claim 1 ). In addition, Mault teaches "determining at least one task name from the text of 
the at least two voice commands" fl|[0061] "beginning swimming " followed by voice 
recognition and activity monitoring inherently requiring task identification from text). 

Regarding claim 10, Mault teaches everything claimed, as applied above (see 
claim 8). In addition, Mault teaches "wherein the step of determining at least one task 
name comprises finding the at least one task name in the text" (see claim 8 rejection 
this operation supported by an activity monitoring application will require locating 
"swimming" in the data). 

Regarding claim 11, Mault teaches everything claimed, as applied above (see 
claim 8). In addition, Mault teaches "wherein the step of determining at least one task 
name comprises associating at least one task name to said time interval between the 
first and second times, wherein the at least one task name is not in the text" flf[0061] a 
variety of activities with start and stop times can be classified as "exercise" and used to 
accumulate activity points). 
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Regarding claim 13, Mault teaches everything claimed, as applied above (see 
claim 8). In addition, Mault teaches "the step of packaging the first time, second time, 
and one task name from the at least one task name into a time increment" (1R0061] a 
variety of activities with start and stop times can be classified as "exercise" and used to 
accumulate activity points). 

Regarding claim 27, this claim has limitations similar to claim 19 and is rejected 
for the same reasons. 

Regarding claim 28, this claim has limitations similar to claim 24 and is rejected 
for the same reasons. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 3-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mault in view of Yahagi et al. (U.S. Patent 4,984,274), hereinafter referred to as Yahagi. 

Regarding claim 3, Mault teaches everything claimed, as applied above (see 
claim 1 ). In addition, Mault teaches "the speech data comprises a time stamp" 
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(1f[0061]), but Mault does not specifically teach that "the step of determining a first time 
comprises: determining an offset time between the time stamp and a time when the first 
voice command is spoken; and determining the first time through reference to the time 
stamp and the offset time." However, the examiner contends that this concept was well 
known in the art, as taught by Yahagi. 

In the same field of endeavor, Yahagi discloses a time measuring circuit using 
speech recognition (abstract). In addition, Yahagi teaches the use of a stopwatch 
where the times are corrected for delays resulting from speech recognition (col. 8, lines 
15-37). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Mault by specifically providing the features, 
as taught by Yahagi, because it is well known in the art at the time of invention for the 
purpose of providing a more accurate timing function when using speech recognition 
(col. 1, lines 44-62). 

Regarding claim 4, Mault teaches everything claimed, as applied above (see 
claim 1). In addition, Mault teaches "the speech data comprises a time stamp" (H[0061 ) 
including a beginning and end time), but Mault does not specifically teach the following: 
• the step of determining a first time comprises: determining an offset time between 
the time stamp and a time when the first voice command is spoken; and determining the 
first time through reference to the time stamp and the offset time; and 
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• the step of determining a second time comprises: determining a second offset time 
between the time stamp and a time when the second voice command is spoken; and 
determining the second time through reference to the time stamp and the second offset 
time. However, the examiner contends that these concepts were well known in the art, 
as taught by Yahagi. 

In the same field of endeavor, Yahagi discloses a time measuring circuit using 
speech recognition (abstract). In addition, Yahagi teaches the use of a stopwatch 
where the times are corrected for delays resulting from speech recognition and include 
the operations of "start" and "stop" (col. 8, lines 15-37). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Mault by specifically providing the features, 
as taught by Yahagi, because it is well known in the art at the time of invention for the 
purpose of providing a more accurate timing function when using speech recognition 
(col. 1 , lines 44-62) and in this case resulting in a more accurate determination of an 
interval. 

Regarding claim 5, Mault in view of Yahagi teaches everything claimed, as 
applied above (see claim 4). In addition, Yahagi teaches: 

• the step of determining the first time through reference to the time stamp and the 
offset time comprises the step of adding the offset time to the time stamp to determine 
the first time (col. 8, lines 15-37, the time delay is corrected); and 
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• the step of determining the second time through reference to the time stamp and the 
second offset time comprises the step of adding the second offset time to the time 
stamp to determine the second time (col. 8, lines 15-37, the time delay is corrected). 

Regarding claim 6, Mault teaches everything claimed, as applied above (see 
claim 1 ). In addition, Mault teaches "the speech data comprises first and second time 
stamps" (H[0061] the time stamps allow for the determination of the duration), but Mault 
does not specifically teach the following: 

• the step of determining a first time comprises: determining a first offset time between 
the first time stamp and a time when the first voice command is spoken; and 
determining the first time through reference to the first time stamp and the first offset 
time; and 

• the step of determining a second time comprises: determining a second offset time 
between the second time stamp and a time when the second voice command is spoken; 
and determining the second time through reference to the second time stamp and the 
second offset time. However, the examiner contends that these concepts were well 
known in the art, as taught by Yahagi. 

In the same field of endeavor, Yahagi discloses a time measuring circuit using 
speech recognition (abstract). In addition, Yahagi teaches the use of a stopwatch 
where the times are corrected for delays resulting from speech recognition and include 
the operations of "start" and "stop" (col. 8, lines 15-37). 
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Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Mault by specifically providing the features, 
as taught by Yahagi, because it is well known in the art at the time of invention for the 
purpose of providing a more accurate timing function when using speech recognition 
(col. 1, lines 44-62) and in this case resulting in a more accurate determination of an 
interval. 

6. Claims 12 rejected under 35 U.S.C. 103(a) as being unpatentable over Mault in 
view of Kmack et al. (U.S. Patent 6,304,851), hereinafter referred to as Kmack. 

Regarding claim 12, Mault teaches everything claimed, as applied above (see 
claim 8). In addition, Mault teaches "wherein the at least one task name comprises two 
task names, a first task name associated with a first of the voice commands and a 
second task name associated with a second of the voice commands" flf[0061] e.g., 
begin/end stationary bike, begin/end swimming), but Mault does not specifically teach 
the following: "wherein the first and second voice commands occur adjacent to each 
other in time, wherein the first and second task name are different, and wherein the 
second voice command is assumed to end a first task corresponding to the first task 
name and start a second task corresponding to the second task name." However, the 
examiner contends that these concepts were well known in the art, as taught by Kmack. 

In the same field of endeavor, Kmack discloses a portable date collection system 
that includes the ability to perform time studies (abstract). In addition, Kmack teaches 
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that the end of one recording event can trigger the beginning of another recording event 
(col. 5, lines 15-30). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Mault by specifically providing the features, 
as taught by Kmack, because it is well known in the art at the time of invention for the 
purpose of providing a more accurate timing function and more easily creating a 
succession of activity records for successive activities (Kmack, col. 5, lines 27-30). 

7. Claims 14, 15, 17-19, 22, 24, 26, 29, and 30 rejected under 35 U.S.C. 103(a) as 
being unpatentable over Mault in view of well known prior art (MPEP 2144.03). 

Regarding claim 14, Mault teaches everything claimed, as applied above (see 
claim 8). In addition, Mault teaches: 

• wherein the at least two voice commands comprises a plurality of voice commands, 
(see below) each of the voice command times associated with one of the plurality of 
additional voice command times fl|[0061] timestamps); 

• converting each of the plurality of voice commands to text (1J[0058] change voice 
message into typed text); 

• determining a plurality of task names from the text fl|[0058] interpret the message, 
H[0061]swimming, eating etc.); 

• associating a task name with two of the first time, second time, or additional plurality 
of voice command times fll[0061] "begin swimming" "end swimming...", timestamps); 
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• creating a plurality of time increments, each time increment comprising two times of 
the first time, second time, or additional plurality of voice command times and a task 
name (1f[0061] a variety of activities with start and stop times can be classified as 
"exercise" and used to accumulate activity points); and 

• storing the plurality of time increments (1J[001 5] access at a later time implies 
storage; fl[0017] ability to download implies storage;. 

But Mault fails to specifically teach "wherein the at least two voice commands 
comprises a plurality of voice commands, wherein the at least one task name 
comprises a plurality of task names, and wherein the method further comprises 
the steps of: determining an additional plurality of voice command times, ..." 
However, the examiner takes official notice of the fact that the use of multiple terms to 
mean the same thing (synonyms) was well known in the art. 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Maul to support this feature, because a 
speech interface that recognizes multiple words meaning the same thing is more natural 
and easier to use. 

Regarding claim 15, Mault discloses an activity monitoring device using speech 
recognition that includes the following steps: 

• accessing speech data comprising a plurality of time stamps and speech (Fig. 4, 
Audio Input, item 95; H[0015] monitoring device to record time and duration; H[0017] 
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ability to download audio files with inherent time information; U[0061] timestamps with 
audio); 

• converting the speech to text flf[0058] conversion into a typed message); 

• composing a plurality of voice commands from words in the text, ... flj[0061] "begin 
swimming" etc., voice recognition may be used with inherent "phrase grammar rule"); 

• determining a time associated with a speaking of each of the voice commands 
(1f[0061] timestamps); 

• determining a plurality of tasks, each task associated with at least one of the times 
and at least one of the voice commands fl|[0061] various activities, "stationary bike" 
"swimming"); 

• determining a plurality of time increments, each time increment comprising one of 
the tasks and at least one of the times flf[0061] a variety of activities with start and stop 
times can be classified as "exercise" and used to accumulate activity points); and 

• storing one or more of said time increments fl|[0061] the calculating of "time 
increments" has inherent storage). 

But Mault fails to specifically teach "each voice command corresponding to a 
phrase grammar rule." However, the examiner takes official notice that the use of 
phrase grammar rules during speech recognition was well known. 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Maul to support this feature, because the 
application of such techniques can improve the accuracy of the results. 
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Regarding claim 17, this claim has limitations similar to claim 3 and is rejected 
for the same reasons. 

Regarding claim 18, this claim has limitations similar to claim 7 and is rejected 
for the same reasons. 

Regarding claim 19, Mault discloses an activity monitoring device using speech 
recognition that includes the following: 

• a memory that stores computer-readable code (Fig. 4, item 92); and 

• a processor operatively coupled to the memory (Fig. 4, item 88), the processor 
configured to implement the computer-readable code, the computer-readable code 
configured to: 

• access speech data (Fig. 4, item 95); 

• recognize at least two voice commands from the speech data, each voice command 
occurring at a different time fl|[0061], "begin swimming," "end swimming"); 

• determine a first time associated with a speaking of a first of the voice commands, 
wherein said first voice command identifies a start of a time interval fll[0061], 
Timestamps); 

• determine a second time associated with a speaking of a second of the to voice 
commands, wherein said second voice command identifies an end of a time interval 
fl[[0061] "end swimming" timestamps); 
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• convert each of the at least two voice commands to text (U's[0058], [0061] voice 
recognition converts voice to typed message) and 

. determine text versions of the at least two voice commands ( U's [0058] typed 

message) by comparing words in the text with phrase grammar rules (use of voice 
recognition inherently includes pattern matching approach); and 

• storing data identifying said time interval and data identifying one or more of said 
first voice command and second voice command fll[0061] recording activity and 
calculating duration implies/requires storage; fl[001 5] for later recall; 1|[0017] transfer 
and remote processing [and access] imply storage). 

But Mault fails to specifically teach "determine text versions of the at least two 
voice commands by comparing words in the text with phrase grammar rules." 
However, the examiner takes official notice that the use of phrase grammar rules during 
speech recognition was well known. 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Maul to support this feature, because the 
application of such techniques can improve the accuracy of the results. 

Regarding claim 22, Mault in view of well known prior art teaches everything 
claimed, as applied above (see claim 19). In addition, Mault teaches "the system further 
comprises a digital personal recorder and wherein the computer-readable code is 
further configured to receive the speech data from the digital personal recorder" 
fl[[0017] can download data to a remote computing device). 
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Regarding claim 24, Mault discloses an activity monitoring device using speech 
recognition that includes the following: 

• a memory that stores computer-readable code (Fig. 4, item 92); and 

• a processor operatively coupled to the memory (Fig.4, item 88), the processor 
configured to implement the computer-readable code, the computer-readable code 
configured to: 

access speech data comprising a plurality of time stamps and speech (Fig. 4, item 
95; U[0058] digital recording of speech with time and location information); 

• convert the speech to text (fl's [0058] [0061] voice recognition into text); 

• compose a plurality of voice commands from words in the text, ... fll[0061] voice 
commands); 

• determine a time associated with a speaking of each of the voice commands 
(fl[0061] timestamps); 

• determine a plurality of tasks, each task associated with at least one of the times 
and at least one of the voice commands fll[0061] multiple tasks and commands: 
stationary bike, swimming, begin/ end); end 

• determine a plurality of time increments, each time increment comprising one of the 
tasks and at least one of the times (fl[[0061] a variety of activities with start and stop 
times can be classified as "exercise" and used to accumulate activity points);and 

• storing one or more of said time increments (^[0061] the calculating of "time 
increments" has inherent storage). 
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But Mault fails to specifically teach each voice command corresponding to a 
phrase grammar rule." However, the examiner takes official notice that the use of 
phrase grammar rules during speech recognition was well known. 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Maul to support this feature, because the 
application of such techniques can improve the accuracy of the results. 

Regarding claim 26, this claim has limitations similar to claim 22 and is rejected 
for the same reasons. 

Regarding claim 29, this claim has limitations similar to claim 19 and is rejected 
for the same reasons. 

Regarding claim 30, this claim has limitations similar to claim 19 and is rejected 
for the same reasons. 

8. Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Mault in 
view of well known prior art and further in view of Kmack. 

Regarding claim 16, this claim has limitations similar to claim 12 and is rejected 
for the same reasons. 
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9. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Mault in 
view of well known prior art and further in view of Yahagi. 

Regarding claim 20, this claim has limitations similar to claim 4 and is rejected 
for the same reasons. 

10. Claims 21 and 25 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mault in view of well known prior art and further in view of Tran (U.S. Patent 5,991,742), 
hereinafter referred to as Tran. 

Regarding claim 21, Mault teaches everything claimed, as applied above (see 
claim 19). In addition, Mault teaches: "store the time increments" flj[0061] the 
calculating of "time increments" has inherent storage), and Mault also teaches additional 
processing of the data collected and file transfer flj's [0017], [0058], [0061]), but Mault 
does not specifically teach "place the time increments into a file having a format suitable 
for importing into a time and billing program". However, the examiner contends that 
these concepts were well known in the art, as taught by Tran. 

In the same field of endeavor, Tran discloses a time and expense logging 
system. Trans system includes the use of a speech recognizer for timing tasks and a 
time and billing program (col. 2, lines 20-67). 

Therefore, it would have been obvious to one having ordinary skill in the art at 
the time the invention was made to modify Mault by specifically providing the features, 
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as taught by Tran, because it is well known in the art at the time of invention that an 
activity monitor that includes the ability to use a time and billing program provides 
information integration advantages (Tran, col. 2, lines 5-17). 

Regarding claim 25, this claim has limitations similar to claim 21 and is rejected 
for the same reasons. 

Allowable Subject Matter 
Claim 2 is objected to as being dependent upon a rejected base claim, but would 
be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to V. Paul Harper whose telephone number is (571) 272- 
7605. The examiner can normally be reached on M-F. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Hudspeth can be reached on (571 ) 272-7843. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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V. Paul Harper 
Patent Examiner 
Art Unit 2626 



